This invention relates to the displaying of video with motion where the frame rate of the displayed video is inadequate to fluidly convey that motion.
A well known problem in the art of video communication is the inability to fluidly display motion in a video when the frame rate at which the video is displayed must be either less than or greater than the frame rate of the source video. Conventional solutions to this problem are either to display the video at a frame rate that is mismatched to the source frame rate or to attempt to rationalize the frame rates. Such rationalization may be done by either a) dropping frames, e.g., during transmission or prior to display, when the display rate is less than the source rate, or b) by padding framesxe2x80x94with or without interpolation-when the display rate is greater than the source rate. Such solutions are inadequate because of the resulting poor quality, e.g., jumpiness and lack of smooth notion.
Another problem in the art is an inability to transmit frames from a video source to a video display without loss of frames during transmission. Conventional solutions to this problem are to either 1) buffer frames so that there is a stored reserve of frames that can be shown during times when no frames are received or 2) display the last received frame until the next one is received. Again, such solutions are inadequate because of the resulting poor quality, e.g., jumpiness and lack of smooth motion, as well as the fact that the cost of the memory to implement the buffer is not negligible. Also, the buffer length introduces a transmission delay, which can be disconcerting in interactive video applications such as video conferences or gaming.
We have recognized that, in accordance with the principles of the invention, when there is an inability to convey motion that is occurring in the frames of a video source by a full-frame-rate transmission, a surrogate effect, other than another form of motion, may be used as a compensation technique to better convey motion to a viewer. The surrogate effect employed may be a) fading, b) wiping, c) dissolving, d) blurring, e) enhancing the contrast, f) enhancing one or more colors, g) enhancing the brightness, h) scaling the image, and i) the like. How the surrogate effect is applied to any frame may be a function of one or more video frames. Optionally, more than one effect may be used in combination. Advantageously, a more continuous sense of motion is perceived by a viewer.
In one embodiment of the invention, full-frame-rate video is initially available. A controller monitors the ability to transmit or display full-frame-rate video. In the event that it is determined that, ultimately, full-frame-rate video cannot be displayed to a user, the images of the frames that will be displayed are adjusted to incorporate a surrogate effect that helps to better convey the intended motion to the viewer. In accordance with an aspect of the invention, the surrogate effect applied may be selected as a joint function of the available frame rate and knowledge of the content, e.g., the specific type of motion, intended to be shown to the viewer.
In addition, the same surrogate effect techniques may be employed even when full-frame-rate capability is available, if it is desired to give motion occurring at a first speed within the video the appearance to a viewer of occurring at a second speed. This may be necessary when the desired frame rate is greater than the full frame rate, which may be required for interactive applications, such as simulations or gaming. The controller determines the first speed of the motion and the desired second speed and applies the surrogate effect as appropriate.